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ABSTRACT * 

The Six Performance Characteristics assessment technique provides a 
means for faculty to judge students in a systematic way over time on % 

Livelopmental characteristics wMr 1 , < { iy to their 'performance afci'oss 
disciplines and across com] • 0 teas 14entified as goals of liberal 

learning by Alverno faculty, riptions of six performance character- 

istics were prepared and first tested by faculty on senior^ graduating 
in £he spring of 1978. The characteristics *were integration, inde-* 
pendence, creativity:, sel f -awareness , commitment* and habituality. The 
characteristics were defined by sets of descriptors for the "Beginning 
Student," 4 the "Developing Student," and the "Graduating Student." 
"Pilot stuciy results indicated some discriminating power (students 
graduating with honors were rated significantly bvyi:er than students* 
graduating without honors). ,The following year all students in the 
college were rated to collect Additional information on inter-rater 
reliability, the developmental character of the ratings, and the extent 
to which the six characteristics were differentiated in ratings. 

^Results from the first all-college, administration provided evidence of » 
acceptable inter-rater reliability, and supp6rted the developmental 
character of the definitions through significant mean^ differences between 
classes. While the power of the technique t6 distinguish between 
students at, different levels was demonstrated, it was found that all 
characteristics followed nearly identical patterns, raising, further ques- 
tiris concerning the differentiation between them. 

F Performance Characteristics ratings were conducted on »all classes in 
1979, 1980,. 1981 and 1982^ as part of a comprehensive program validation 
wh^ch included other meaSures\of student performance within the curric- 
ulum, and longitudinal assessments of student development and change using 4 . * 
a batterv of external crite.rion measures (Mentkowski & Strait*, 1983). It 
was confirmed with ratings from the longitudinal study sample of two con- 
secutive entering classes that a single factor accounted for 90% of the 
variance in ratings on each characteristic on three different occasions. 
Using the single factor, it was f^JJ*^ that students were rated at signif- 
icantly higher levels over time, corroborating the cross-sectional 
evidence for the* developmental character of the procedure. The rating 
factor was not assdciated with otfyer college performance measures in the 
longitudinal study when the influences of student backgreuTid and program 
differences were controlled. 'There was however evidence th^t ratings dis- 
criminated between students on academic probation and those who were not; 

irrespective of class Standing. ' 5 

» , n» 

Relationships between the Six Performance Characteristics factor and the 
measures of. human potential revealed .that thp faculty were making judg- 
ments based on a general dimension associated with several external 
criterion measures of intellectual, egc^ and mora^ development. The 
strongest pattern of associations was^found with a measure of Perry's 
^^—Bcheme of intellectual and. ethical development during the college years 
(^Trry, 1970, 1981). The 1 Alverno, faculty is continuing to work with the 
assessment technique, attempting to refine the definitions of several 
characteristics so that a more differentiated pictuije qf student develop- 
ment; may result • v • 
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validating- Assessment techniques in an outcpme-centered 
liberal arts "curriculum: six performance characteristics rating 

«* / ' 

' ■' ' ^■ > — ■«"' 

;: Assessment Committee/Office of Reseal*e4i and Evaluation 

ALVERNO COLLEGE ' 

■ \ * - -~ 

*The Six Performance Characteristic^ Rating : 
A Faculty Rating of Student Performance Characteristics 



The -Six, Performance Characteristics rating, an Alverno College 
assessment technique, provides. s a means for faculty to assess ♦ ■ 
students at regular intervals^ on several - characteristics which . 
apply to her perfoirmance across disciplines and L : across compe- 
tences identified as central objectives of liberal -learning at 
Alverno College- The rationale,- descriptions of Six^Perfoir- 
mance Characteristics, 'and an initial rating, instrument were 
developed by the Assessment, Committee and the Office of „ Research 
and Evaluation in the 1977-78 academic year. 

TftJLsix characteristics identified by Alverno faculty are 
integration, independence, creativity, self-awareness,, commit- 
tment., and "Habitual! ty. The last, habituality, was conceived as 
modifying the^gther five. Original descriptions of the Six 
Characteristics^re included in Appendix A. At the outset, the 
characteristics were developed to.serv6 a particular need for 
criteria used in judging advanced^ students in the performance of 
complex and interrelated abilities required in' their 'major areas 
of concentration. They were to function as general criteria in 
assessing student endeavor at advanced levtels/ After an initial 
rating of graduating seniors in the spring of 1978, it 
decided to expand the rat' ffort to al^ 3ses of sl ^nts 
in the Weekday Cbllege, to uelp further understand the levelop- 
mental nature of the characteristics. An Assessment Committee 
report On the* pilot rating study is presented in Appendix B. 

This paper briefly describes the development, use, «and analyses 
of the assessment technique from its first employment in the 
spring of 1978, through its college wide implementation in the 
1979, 1980, alid 1981 spriiig semesters. ■ The external validation 
of the assessment technique is rfeported^ in terms Of the relation- 
ships ^betwec a the Six Performance Characteristics rating (SP^) , 
other indices of college performance available from' student 
records or being developed by the faculty, and external criterion 
measures of "human potential administered .during >t&is period as * 
part of €n overall evaluation/validation of Alverno's curric- 
ulum (Mentkowski & Doherty, 1977, 1983; Mentkowski; 1980; 
Meiitkowski & Strait, 1983). . 

Several strategies for validating outcomes and new assessment 
techniques were developed into a comprehensive validation model. 
The model incorporates various research >and evaluation methods, 

O ' ' , 



with the ultimate goals of establishing program validity, con- 
tributing to program development, ancj developing a picture of 
adult ^learning and development that can be used to improve 
instruction and assessment in liberal education settlings.* 
Figure 1 displays the several components of the validation 
ntodel. , * * 

The! results reported'in this paper' link faculty ratings on the 
Six Performance Characteristicsvto other internal college per- 
formance, indices (e.g., credit hbiirs and competence^ level units), 
to atiother new Alverno assessment technique, the Xntegrated 
'Competence Seminar, and to external drit^rion measures of human 
potential. Other papers present reisul£s linking the human 
potential measures to the. Integrated Competence SemiJ^r (Assess- 
ment r Commit3rtee/0f fice of Research and Evaluation, 1982)^and to 
the performance indices (Mentkowski 4 Strait, 1983). 

The Rating Instrument and the Procedure * 

\ i 

An example of the revised rating form is presented in Appendix C. 

• A nine puint scale was used for rating each characteristic. The , 
descriptioTis of the characteristics developed by the Assessment 
Committee included descriptors identifying the "Beginning Student," 
th* "Developing Student," and the "Graduating Student." The 

"Mnstruetions^ph the rating form ask thfe faculty member to rate 

' : 3feacb. student using the' entire nine point scale, irrespective f t> 
stu(Sf.ttti> s class atandiri^. The form included a spp * 
comments."" --.A* scale for Habitual* Led ..aor each 

of the five other c 1 cteixsti, ^, mi u. >±.tual. «, ,ingSt*as given 

only in those cases iere the other characteristic was rated, a four 
or above, on the principle" that a ;lower rating precluded the 

% possibility of habituation fceing evident. Each of tjhe five char- *> 

acteristics was rated oh a separate pagg. . , 

f . 

As indicated above, the first use -of the instrument in Spring 1978 
involved ratfhg-only graduating Seniors. In that administration, 
two faculty from a student's major area rated each student and 
then attfived at a consensus rating . Only consensus ratings were 
collected and analyzed. It was from a preview of those initial 
results that the- form and procedure described here we re drawn. 

The procedure was revised so that, for^JunioVs and Seniors, two 
/faculty from the student's major area, "and one from her minor area-, 
would be tnsJcing a rating, for the purposes of collecting inter- 
rater reliability ^ata. Freshmen arid Sophomore^ were rated by at ■ 
least two of their instructors, bu^ these individual ratings were 
averaged rather than subjected to a consensus process?. Specific 



procedures are outlined in Appendix D. 
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Results From the First College-wide Faculty Rating 

The Spring, 1979 data collection was designed to yield improved * 
information with 'respect to the following issues; 

* ' * ' V 

1) Practicality and utility of a c|llege~wide rating ^ 

2) Intpr-rater reliability 

4 3) The* extent to which the Six Performance Characteristics 
f are d&velopmental 
4) "Are the definitions differentiating jam^ng the 

characteristics? To what extent are they inter- 

Imp lementa tion . 

< 

The Spring 1979 rating of Weekday College studen£s included all 
students enrolled. The descriptive statistics cited below demon- 
strate the effectiveness of the, rating procedure, and the Alvernd 
faculty's ability to collaborate on this college-wide task. 

Table 1* indipates that ,91% of all th^jteekday College students • 
were rated, an" extreme ty high percenjia'ge. ; 

Table 1. 

Sample D|escriptipn of Spring, 1979 
Six Performance Characteristics Ratings 



// Enrolled // Rated % Rated 



Freshman 
Sophomore 
Junior 
Senior 



216 


'175 • 


81% 


'209 1 


203 * 


, 97% 


124 


117 


94% 


97 > 


95 


97% 


646 


i — r ~ 

591 


91% 



Table 2 presents the number of instruments on which comments were, 
recorded per class standing. It was assumed that ^f acuity rating 
Seniors of Juniors were more familiar with the students and would 
f> be more able to write comments, as compared tq facult y who grated 

v * Freshmen and Sophomores. Table 2 shows ^ that similar ^^Jentages 

;pf. jcomments / were obtained across all four class standitvgs. This 
a suggests that comments can be 'an effective topi for collecting 

information for a student's file that can be used later for 
^ „ ^ writing narrative trairsqripts. ' 
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Comments by Class Standing 



freshman 
Sophomore 
Junior ' . . 
Sefiior , - 



n f 'L^ a Jr e ii t Comments % Co mment s 



175 

204' 

117 

*<>5 



109 
105 

66 
5i 



62% 
51% 
56% 



The total number of students rato^i was broken • down by discipline 
divisions to get a better understanding of the intradivisional 
procedure for pursuing the rating task. Table 3 specifies the 
number of studerfts who were expected tp be rated and the actual 
number rated (sample total) by division. Overall,; most of the 
divisions were close to .the expected totals. 



Table 3., 



\ 



Number of Student;* Rated by Divis 



sion 



Pi v 1st oi 



2) Behavioral (Science:* 

3) Education 



5) Nursing 

6) Performing Arts 

7) Undecided 









So. 


Jr. 




Sr. 








r r . 


Minor : 


?a jor 


Minor * 




Expec te J 


Total 


7 


12 


9 


10 


8 


7 


Sample 


Total 


* 3 


17 


8 


9 




5 


Expected 


I a 1 


33 


*J0 


4. 7 


15 


U 


lb 


Sjmp le 


Total 












15 


Expected 


Total 


9 


6 




12 


U 




Sample 


Total 


5 


9 


4 


8 


13 


J3 


Expected 


Total 


5 


, 12 


lb 


13 


. 16 


7 


Sample 


Total 


5 




10 


i i 


19 


3 


Ex pec ted 1 


Total 


131 


1 2B 




79 , 




37 * 


Sample 


Total 


110 






71 




55 

""ii 


Expected 


Total 


7 


11 


2 


10 




10 


Sample 


Total 


* 10 


17 


1 


10 


2 


10' 












V 






Expected 


Total 














Sample 


Total 


8 













The rat&d samijl^ obtained is representative not only^across class 
standing but also within divisions. High percentages of minor 
and major ratings were obtained (as compared to the expected 
numbers). The present analysis throws light o;i the Jfcctent to 
which faculty can handl^ such a college-wide instrument rating. 



Inter-rater Reliability of Six 'P erform ance * 
Cha racteristics Ratings 

V 

The question of expert judgment is a major concern in our effort $ 
to validate assessment techniques employed in the college* The. 
Six Performance Characteristics data provided some insignt into 
these questions:* 

1) What is the extent of agreement between faculty ratings 
yof students when ttie faculty doing the rating are 
from the same department? from different departments? 

i * 

Table 4 indicates the number of 'students who were rated by facult 
from a student's major department wirtiin each class standing. 
Freshmen and Sophomores were rarely rated by faculty within their 
major department since the criterion for selecting faculty raters 
for Freshmen and Sophomores was instructors teaching freshman or 
Sophomore courses irrespective of the student's major. 

Tabled. % 

* . 1 * 
Major^ Department Rat ings 



Freshman 10 

Sophomore 42 

Jtmior 97 

Setiior 84 



1 

Note. Faculty raters are members 



of the student's major 

department . - , * 

r 

For Freshmen and Sophomores the inter-rater reliability data 
indicated significant correlations between faculty ratings from 
within the major department as well as between faculty ratings 
from o utside # student's major department. The inter-rater 
reliability coef: f icienks from outside a student's major depart- 
ment were somewhat lower as an overall pattern. (See Table 5.) 
Habituality was excluded from these analyses due to the special 
procedures involved in its application. 

'« » * 
% r*bu 5. \ 

\ ■ 

Inter-rater Reliability Per\ Characteristic tor Fre^haan-Sophoam* Rating 

__ ^ 

IntWftratlon Independenc e Cr eativity Aweren ea a Cociait^gn t 

Faculty Rating* From Within 

Studenta' Major Compared r - .45 40 .52 .J6 . J> 

(Apprcx. 5i Students) -001 .002 .001 .00* .005 



Faculty Ratings Fro* Outaide 
Students' Major Compared. 
(Approx. 200 Students) 



,40 .)0 .38 .78 *38 

.001 .00! .001 .001 .001 



Note, s ■ significance level 



2) To what extent will faculty agree on student's rating 
when a student is rated in both her major and minor 
department? " o 

.,. 3) To what extent is the consensus rating (two faculty 
compare individual ratings and come to consensus) 
f ^ consistent with tRo^g^rage rating (two faculty 

* • submit individual x^tinfes and f they ajce averaged) ? 

One of the curriculum pedagogical objectives is to have students 
function similarly within their majoi^and minor area, thus empha- 
sizing the. concepts^ of transfer' and internalization of abilities. 
The Six Performance Characteristics bating yielded data which 
enabled us to compare faculty judgment on major and supporting * 
ar.ea and .explore the consistency of students* performance in both 
their major aqd minor areas. 

yable 6 -shows that the* inter-rater reliability between Juniors! 
an0 Seniors' major &nd 'minor ratings is signif icahtly consistent. 
Similar results were obtained with regard to the major department^ 
agreement. The highest agreement correlations were obtained when 
con&ensus rating wap compared to the average rating given by each 
of two' faculty from the student's jp^or. department. ~ Such high 
correlations suggest that the consensus procedure can be eliminated 
from the rating, since averaging of two ratings from two faculty in 
tpe student's major department wil£ provide almost idenlHcal 
results. * 

TaMe 6. 

Inter -rater Reliability Per Characteristic 
For Juniors and Seniors 



* ~ Integration Independence Cr eativity Avarapjj& C cTsaitr-ent 

Average "Major" Rat in* Compared 1 ▼ 

to Average "Minor" Rating r * .47 .52 .52 * * .39 .iO 

(Approx. ilO Student*) ,001 .001 .001 .001 .001 



Conaenaus Conpared to 

r* Average "^tajor" Rating % r • .94 i .94 .92 ,91 

(Appro*, 180 Students) * s » .001 .001 ;00l ,001 .001 



Note , _s « significance level' 

Are the Six Performance Charac teristics Developmental ? 

This question examines the^extent to which faculty's interpret 
tation of the characteristics is consistent w J th the intended 
developmental meaning of the definitions. If ratings are 
found to be developmental, the data should indicate that: 

1) Student ratiogs within each class standing fall 

along the full scale range from 1 to 9. 

2) The majority of students demonstrate a develop- 
/ mental pattern, l:e. the majority of Freshmen 

will be rated lower than the majority of 
Sophomores, etc., per characteristic. 
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' Figure 2 represents the distributions of percentages of • 
students as they were rated on the 9-point scale per 

characteristic. 

• • * ~ .-■ •* 

~"^H6aQh class distribution is presented by a solid line, 

whereds, the mode of each class standing is indicated by a 

♦ vert ic^a^ broken line. The five graphs demonstrate a con- 
sistent developmental pattern. Th$ modal point for Fresh- 
men on the scale is 2 to 3 (beginning)*, for Sophomores is 
3 to 4 (developing), for Juniors is 4 to 5 (developing), 
- and fdr Seniors is 7 to 8 (graduating) . The graphs also 
indicate that the whole scale was used for each class 
standing. Thus, faculty recognized individual differences 
within each class standing. 

The Freshmen, Sophomores and Juniors seem to progress upward 
in a consistent manner, whereas the Seniors are making a* 
leap at>d are far ahead on the scale. 

Table 7 indicates the mean rating for each class standing 
per characteristic.- ?k discriminant analysis ipdicated 
significant differences vertically (across class standing) 
but not horizontally (acpqss characteristics). Thus, 
although the characteristics were proved to be develop- 
mental, ratings across" the characteristics are similar. 

Table ?. 

Mean* and Standard Deviation* Per Characterise 
. Within Each Class Standing 



Number of . _ . 

Instruments Integration Independence Creativity Awareness Commitment 



Freshman 288 



M - 3.2 3.3 ' 3.2 3.3 3.5 

SD - 1.5 1.6 . 1-6 1.6 1.7 



Sophomore 406 H ■ 3.9 4.1 3.9 4.1 4.1 

*o - t.5 1.6 1.5 1-6 1.7 



Junior 



19| HV* 5.3 5.4 5.2 5.3 5.8 

SO - 1.5 1.6 ' 1.4 1.5 1.3 

167 M- 6.6 7.1 6.7 6.7 7.2 

SD • 1.8 1.6 1.6 1.7 1.7 
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Results from the* pilot study of ratings for graduating 
Seniors (see Appendix B) included findings of group 
differences between, majors and between Seniors gradua- 
ting wiYfr or without honors. The findings of class 
differences corroborate that preliminary evidence,, for 
the discriminating power of the rating procedure: 



/ 
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Are the Definitions Differentiating 
among the Characteristics ? 

Intef-cqrrelations among the characterise ids disregarding class 
'standing indicated that the characteristics are highly inter- 
related (Table 8). 

* ' '.^Table 8. f 

Table of Int?rcorrelation for All the Six Performance 
^ Characteristic Data Disregarding Class Standing . * 

Integration Independence Creativity Rareness Coranitnent . 







1 * 


2 


3, 




1 












2 












3 




.767 


'.775, 






4 




.815 


( .804 


.777 




5 


# 


.79 


> .776 


.712 


.816 



. s — ~: ~ 

Figure 3 demonstrates clearly how the class standing • 
distributions across' characteristics form almost a, straight 
line, even though there are significant differences between 

classes. 
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Figure 3. Distribution . of ratings across Sjix' Performance CharfTtte 
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Faculty Perceptions of the \ * 

Rating Process ' 

Aftei^rating students, faculty completed q ues tionnaires evalua— 
ting the rating process. The questionnaire, with quantitative 
summary data, is presented in Appendix *E. 



Faculty were asked cto indicate^ which of five kinds of evidence, 
ot sources or judgment, they used in rating, and to rank order 
the ones they used. The five source^ given were observed behav- 
ior, performance on assessments, sense of the student, / 
conversation with stTJ^entr, and handbook definitions. Virtually 
all* of the 39 faculty who responded indicated they used all -five 
sources. Observed behavior was ranked first in importance, per-,, 
formance On assessments was second, sense^oF student and 
conversation witR^student tied for third, and the definitions 
were ranked as least important. ^ 

When asked how clear and understandable the handbook definitions, 
were, the faculty gave similar average ratings to each character- 
istic. Oh a scale of 1 to 7 with 7) most clear, all mean ratings 
'were at or just below 5. Independence and Commitment were rated 
as slightly more clear and uqfferstandable* than Integration^ 
Creativity, Awareness, and Habituality, all of which received 
the same mean rating. 

Faculty who responded goitferally felt the process would have been 
more effective if they had had more advance notice of the rating 
in which to make mental notes on students they would be asked . to 
rate. There was.no single precise procedure used by all faculty ^ 
in reaching concensus, but mo9t indicated they discussed individ- 
ual rating^ first and then chose a single rating or averaged 
their ratings. v \ 
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Validating the Six Performance Characteristics toting 

'in Relation to Other Studeht Performance 

Measures in the Longitudinal Study 
* 

Longitudinal data were collected on , two consecutive classes ?*» 
entering in 1976 and 1977. The measures of/^uiman potential, 
identified in Table 9, were administered ort three occasions: at 
entrance, twq years later, and three and one-half years after 
entrance. Table 10 shows the temporal relationships among 
several components of the , comprehensive validation model. The 
first all-college Six Performance Characteristics rating in the 
spring of ^979 Occurred^in the same academic year as the 
second longitudinal assessment of Weekday, College students who 
erftered in 1976. Ratings in the following two years, 1980 and 
1-981, effectively provided two ratings! for each longitudinal 
Weekday College cohort, roughly coinc iding with the second and j 
third longitudinal assessments on human potential measures. 

ii- 

v 17 



Measures 



Table 9. 

of^Human Potential From the Longitudinal Study 1 ^ 



HUMAN POTENTIAL MEASURES : COGNITIVE DEVELOPMENT 
$ » > " ■ 

Test of Cognitive Development (Rentier et al., 1976; after Piaget) % 

By' having a s'tudent work a serj.es of problems and provide reasons for his 
or her answers, this instrument measures ^student's cognitive activity . 
based on Piagef's stages of cognitive development. 

SentencCcomoleUon Test (Loevinger, et al., 1970) 

This instrument provides a measure of an individual stage of ego 
development. "Ego" here is defined as one's "style of lif ^L^t Jo. ' s 
'personality, individuality, the method of acing problems, P M Voices 
self and, tHe| problems of life, and the whole attitude toward making choices 
in all life/spheres. ^ 

Defining Issues Test (Rest, 1979) 

v 

- Rest's instrument (based on Kohlberg's theory of moral development) 
provide am a sure of an individual's moral development in • "J^^** 
bv analyzing the relative importance attributed by a person to principled 
^"considerations. A person attributes importance to 

given for resolving a particular moral dilemma, and then rank orders them. 
Measure of Vocational. Edutational^mdJ^^ 

Widick, 1975; »ow titled: Measure or Intellectual Development, after Perry^ 

This measure of the Perry scheme of ^^^^££2^ 
asks students to write 'hree essays ^ their best c^s^ t 
their career. It assesses the progress tne conege »u ^ 
movement on the Perry scheme. 

HUMAN POTENTIAL MEASURES: GENERIC ABILITIES 

Test of Thematic Anal ysis* (Winter, 1976) 

This instrument consists* of two sets of stories "udent^are jsked to 
compare thematically. this "thematic ana js s J scored ceo rdi 
twelve categories of critical thinking. This test is based o n and 

of cognitive development defined as the ability to analyze ne _ ability 
to synthesize new concepts baaed on this .information, and ref lects tn e a Y 
to integrate information into one's own cognitive at ructure. ^As the cogmt 
structure grows, so does the ability to think critically, to make g 
argument and to reason inductively. 
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Table 9 continued. * 



Picture Story Exercise* -(Scored' for Stages of Adaptation (Stewart, 1977), * 
Self-Definition (Stewart & Winter 1,9741) ; and Achievement (McClelland, et al. , 
19,5.3, Affiliation (Atkinson, 1958), and Power (Winter, 1973) motives.) 

This instrument, modeled on the Thematic Apperception Test, may be 
used to assess a variety of abilities. The instrument requires the student 

— to write nai rartvgg to "sT\ pictures. One is "self-def inition" which encon)- 
passes the way one thinks about; the wotld and one's self, the way one reacts 
to new information", and the wa/ one behaves (S'tewart and Winter, 1974). ' - 
Pfeople^/Lth\ high cognitive initiative are not'' only able to think clearly, 
but also to \reason from problem to solution, .and to propose and take effective 

. action on th^ir own/ This instrument is also used to assess Need for Achieve- 
ment (McClelland, et *1. , 1953) Affiliation (Atkinsoij, 1958),, Power (Winter, 
1973), and Stages of Adaptation, a measure of ego development (created by 
Stewart, 4977). K ' . I 

* I » \ • / ^ 

u Wat son-Clase r; C riti cal Thinki ng Appraisal (Watson and Glaker, 1964) 



J- 



This instrument measures several components of critical.. thinking: 
Inference, Recognition of Assumptions, Deduction. 

\ 

Learni ng Style I nventory* (Kolb, 1976) 4 x 

The Learning Style^ Inventory is a measure of individualxlearniitg styles 
which affect decision-making and problem-solving. The four styles are 
Concrete Experience, Reflective Observation, Abstract Conceptualization, and 
Active Experimentation. The instrument requires the student to rank order 
descriptive* statements about her mode" of learning. 

. * \ 



^Available £rj6m McBer and Company. 

For a more derailed description of the measures and tjLeir 

use in the /longitudinal study, see Mentkowski, M. & «rait, M. 

A Longitudinal St udy of Student Change in Cognitive Development 

anlJgenerl c Abilities in an Outcome-Centered Liberal Arts Curriculum . 

Milwaukee, Wi : A4verno Productions, 1983. 
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Table 10, ', 

Design for the Administration of Human Potential Measures and Student Perception 
Measures for Longitudinal and Cross-Sectional Studies of Student Outcomes 
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Note . See Figure 1 for overview of components of the program validation model with measures, 
Student Perspectives Interview (SPI). data were collected" on a'subsample of students 
participating in the administration of the Human Potential Measures (HPM), but all cdfyteted 
the Attftude Survey (AS) and Careering Questionnaire (CQ). All' Weekday College students completed 
the Integrated Competence Seminar (ICS) and were rated by faculty on the Six Performance 

Characteristics (SPC), 



/ . . . 

^Additionally, Six Performance Characteristics ratings c&n be 
compared with performance on the Integrated Cbmpej&ence Seminar, 
an^ with other college performance variables collected in the 
longitudinal study : the number <^ semesters attended during 
each of the two intervals betvfeeru-Assesgment , ' the number of 
credit hours successfully completed in each interval, and the 
number of competence leve3r units achieved in each interval. 

... . . . J 

* The Six Performance Characteristics rating was not conducted 
with students in the Weekend College. The student population 
described in this report was. relatively homogeneous with respect 
to such' variables as age, marital status,, and religious Affilia- 
tion.^ A large majority of students were 17 to 19 ye^rs old at 
entrance, not mSrrijid, and attending full time. 7 

Seventy- percent identified themselves as Catholic, two-thirds 
were commuting, <*and two-thirds were intending to major* in 
nursing* Edch -of these separate background or pr©g«a*i^ character- 
istics is highly interrelated with tTie others, a fact which 
indicates there was £ typical atudent described by most or all »of 
these classifications, and a^OTiall percentage of atypical stu- 
dfentls who were likewise noMLdentif fed by most or all of these 
classifications. For example, if a student were older t^n 19 at 
entrance, there was ^Iso a good chance ^that she was married * j 
attending part-time, and nqt Catholic. This simple inverse por- / 
% trait does not hold for, r£si£ence and jnajor hoi/evejr. Atypical J 
students were most likely* nursing students and wer^ commuting. 
Otherwise typical students provided the sm^ll percentages of -j 
^majors in ^traditional liberal 'arts disciplines and resident stuff 
^dents. A more detailed description of: the student popul&ti6n / 
can be found in' the fongitudinal study of student change on th^ 

human potential measures (Mentkowski & Straife, 1983) . / 

✓ • ■ // 

Relationships Between Six Performance // 
Characteristics Ratings and Student // 
Background and Program Variables, / ' 

in the Longitudinal Study ~ / 

* if 

Not sihrprisingly, due to 'the homogeneity of the sample, th^e 
were few statistically significant correlations between ratings 
afia student background or program variables . There wfits. a 6m£ll, ^ 
but significant correlation between ratings coinciding wityh 
second assessment, and students 9 high" school grade point / 
* averages Cr = .220, n « 110 ,/j> - .01). There H ere no signifi- 
cant correlations With age, Religious affiliation, parents' 



education or occupation', mar 
experience.,, 



tjal 



status or prior colleg 



There was also a sizable coiyelation between 1 ratings ahd entrance 
cohort (_r = .481, n 53 129, £<.001VT NThe -interpretation of this 
association is made difficult by Jme fafct that the ratings for 
the two cohorts were conducted alfull ye\ir ap^rt and under 
changing procedures. Thus, the porrelatiim may indicate a 



significant change in ^faculty judgements. On the other hand* the 
measures of human pcrtenti^l also show significant cohort effects 
in the study of change (Mehtkowski & Strait, 1983), so the cor- 
relation may, at least in part, reflect actual group differences 
in the two entering classes. ^ 

There was a high correlation between repeated ratings a year apart 
(:r=-.667, n = 125, £ < .001), indicating stability in ratings. 
Variability in ratings coinciding tSrith third assessment not ex- 
plained by ratings the preceding year was not related to any 
background or program characteristics.^ 

Analysis of* Six Performance Characteristics 
Ratings for the Weekday College 
Longitudinal Sample > ^ 

I " - } 

The combined classes provided a longitudinal sample of 136 
students. Six Performance Characteristics ratihgs were available 
for 129 Qf those students at* second assessment* and 131 of Jfctva 
students; at third assessment. . In the analysis of results from? 
the first all-college rating In 1979* it was shown that the 
characteristics were developmental, based on the significant mean 
differences and rating distributions among acad^jnic classes. ** At 
the 'same time, it was shown that the intercorrelatlons among the 
laracteristics «pre very high, i.e.* each characteristic seemed 
to be showing a nearly identical pattern- Ratings data from the 
longitudinal sample provided strong evidence supporting the 
implication that the five ratings (discounting. the special base 
of the Habituality rating) were redundant indices of a single, , 
global Judgment. * i 

Table 11 presents intercorrelatlons of the f ive^characteristics 
for the same students on three occasions. The consistently 
high correlations indicated that a single underlying dimension 
was being rated rather than six differentiated characteristics. 
This interpretation was further confirmed by factor analyses #it 
each Occasion which resulted in. a single principal factor 
accounting for approximately ninety per cent of the total vari- 
ance "among ratings. 



On the basis of this strong evidence for a single underlying 
factor, further analyses of tfce longitudinal data* were carried 
out using single factor score instead of the five or six 

separate characteristic scores • 

.Using th^ single facto^core, the evidence of class differ-\ 
ences in ' the cro»s-sectional analysis of 1979 ratings w^o-—^ / 
corroborated by evidence of significant change over ^ time i^n tW 
longitudinal study. Figure 4 shows the mean rating ^f studentk 
ovep three occasions of assessment. Similar change* oyer 4:ime 
was found for the 1976 entrance cohort between 1979 'bnd 1980 
ratings. v • - ^ 
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. Table '11. * \ 

* ' -I \ ' •■ 

Intercorrelations 6f Si?; Performance Characteristics on Three Occasions t 

' Entrance Cohort) \^ / t 



Chaiacter.istics * Spring '79 - Spring '80 / Spring ! 81 

•Integration- Independence ..90 . ..90 .86 

Integration-Creativity .68 .87 .79 

'Integration-Awareness # ..79 .85 *80 

Intagrat ion-Commitment .75 .81 ,69 

* i * 

$ * ' * 

Independence-freativl^ .73 .88 t .83 



Independence-Awareness 



S 



.83 .85 • .82 



Independence-Commitment .78^ .83 # . , % .79 

Creativity-Awareness 



f 

.72 .85' .83 



Creativity-Commitment > .67 .72 ,7 ** 

Awareness-Commitment ' r .79 .78 .82 



Note^ Correlations based 4m 75 students. 
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Figure 4, Change in Mean Rating Over Three Occasions of Assessment 
(1977 Entrance Cohort). 
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# Relationships Between the Six 
Performance Characteristics 
Ra tings Factor and Other College ' 

Performance Measures % 

v — r~ ; , - „ 

There. were no significant partial correlations .between the Six 
Performance Characteristics. rating factor and other college 
performance measures, controlling for variability attributed to 
student background and program differences'* For the combined 
Weekday College cohorts, the number v of semesters attended 
between entrance and the secAnd longitudinal assessment was 4 
related only to students' /major field of study Controlling 
for this source of variance, there was no relationship 
between number of semesters attended and 'faculty ratings. 
Credit hours accumulated during the same period were correlated 
with students! «ge, major field, and full-time status. Con-.* 
trolling for these differences, there Vas* no relationship 
between faculty r<atings and the number of credit hours success- 
fully completed. Competence level units achieved were' 
associated with students' high school grades* their mothers 1 
occupations, and major field and full-time status. The partial, 
correlations between competence level^units .and faculty ratings 
were not significant with these variables Controlled, This 
finding corroborated earlier evidence of no relationship 
between^ ratings and rate of learning in the an^iysi's of the 
first all-college sample in 1979. 




The other measure of student performance developed by Alverno 
faculty, the Integrated Competence Seminar , * fs covered in more 
detail in another report (Assessment Committe-e/.Of fice of 
Research and Evaluation, 1982) . Scores t on that performance ; 
measure were also not related to faculty ratings, when other 
sources of variance were controlled . * - 

While the Six Performance Characteristics rating factor were 
not associated uniquely with semesters, credits, or competence 
level units in the longitudinal study, one piece of evidence 
was supportive of the discriminating power of the faculty 
ratings shdwn earlier, in .relation to major field,! honors, and 
academic classification. A group of students on academic pro- 
bation'in* the fall of 1981 were found to have received 
significantly' lower facuUty ratings the preceding spring than 
students not on probation, tn each of these instances of 
group .differences ho 3 w£ver„ it should be noted ',hat control 
variable^ were not available as they were in the longitudinal 
study* 



( 



. , • 19 
.. v 



ERIC " 0 



Relationships Be twee n the Si x 
P e r f o rmanc e Cha racteri st ic s 
Rating Factor and Huma n Potential 
Measures In the Longitudinal Study 

if"' 



As stated earlier, the repeated administrations of the human 
potential measures at entrance, two years later, and three and 
one-half years later, and the repeated faculty ratings on the Six 
Performance Characteristics afford multiple sets of correlations 
in different temporal relationships. When so many correlations 
are generated, the possibility is much greater that the usual 
conventions for assigning statistical signififcance will be met 
by pure chance. Therefore*, the following results emphasize 
patterris of statistically significant relationships in tha data 
rather than focusing on individual bivariate relationships . 

Examining the many possible relationships between the Six Perfor- 
mance Characteristics factor score and the human potential 
measures, our general conclusion .was that a cognitive- 
developmental factor was being evaluated by the faculty. 
Table 12 lists those measures that showed significant relation- 
ships to the faculty rating in the left column, and those that 
didn't in the right column. Significant relationships existed 
with all of the human potential,^ measures specifically designed to 
assess very broad dimensions of cognitive development: a mea- 
sure of Perry's scheme of intellectual and ethical development in 
the college years (Perry, 1970, 1981), Loevinger's (1970, 1976) 
measure of ego development, a measure of Kohlberg's theory of 
moral development (Rest, 1979), ,and a measure of cognitive 
development as defined by Piaget. The simple correlations are 
presented in Appendix F. ~ 

When all student backgraundj^and program differences related to 
the human potential measures and the faculty ratings were con- 
trolled, the few statistically significant partial correlations 
still supported^ this general conclusion. Faculty ratings 
coinciding with second assessments were significantly correlated 
with entrance assessments from the three essays measuring Perry's 
scheme, of intellectual and ethical development (" Bes t, Class" 
essay, r » .215, n ■ 98, £ ■* .032; "Decision" essay, r * .J23A, 
n » 96 r £ * .021; and "Career" essay, r = .250, n 78/ 
.025). . * 



Furthermore, faculty ratings coinciding with second assessments 

ere significantly correlated with change on some of the human 
potential measures during both intervals* Change during^ the 
first two years on the Perry "Best Class" essay unexplained by 
differences at entrance was Correlated with faculty ratings, as 
was change on Rest's measure of moral development, Stewart's 
Receptive Stage of Adaptation, and the WatsonKJlaser Critical 
Th inking" ApprSIsal^ During the second two years, faculty ratings 
were correlated with the ?erry "Decision" essay and-Rest's 
measure of moral development. 
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. j Table 12. 

'Relationships Between Six Performance 
Characteristics Factor and Human Potential Measures 



Significant Re lations hip : 



No Sig nificant Relationship 



Measure of "Vocational, Educational 
' and Personal Issues (Perry) 

Sentence Completion Test of Ego 
Development (Loevinger) 

Defining Issues Test (Rest) 



Test of Cognitive Development 
(Piaget) 

Picture Story Exercise - Stages 
of Adaptation (Stewart) 

Critical Thinking Appraisal 
(Watson-Glaser) 



Picture Story Exercise - Self- 
, Definition (Stewart) 

Picture Story EMtrcise - 

Achievement Motivation (McBer) 

Picture Story Exercise - 

Affiliation Motivation (McBer) 

Pvwrure Story Exercise - Power 
Motivation (McBer) ^ 

Learning Style Inventory (Kolh) 



Test of Thematic Analysis (Winter) 



Notef ThSse relationships represent generalizations frbm to(v> correlation 
" ~ coefficients derived from two faculty ratings with second and 

third assessments of human potential measures. Approximately 120 
students* were included in each analysis. 



2i 

: . 28 



From these patternVof relationship, we concluded that the 
faculty were sensitively rating differences and growth^ in 
student development, but in a global fashion^ not according 
to five or six differentiated characteristics. 

Summary and Di scussion 

» 

Alverno College faculty have developed an assessment technique 
that can be used to rate students on important characteristics 
of their performance across competence areas and disciplinary 
lines. The faculty demonstrated that they could successfully 
implement such an institution-wide rating procedure. 

t, 

Results from the first college-wide faculty gating indicated 
that an acceptable level of inter-rater ^reliability was achieved 
irrespective of whether the faculty rater wa&Jfrom the student's 
major or minor department, or was an instructor in a single 
class (in the case of Freshmen and Sophomores)* It was deter- 
mined that the consensus process accomplished the same as* 
averaging the ratings of two faculty, thus permitting a 
simpler and less time-consuming process in subsequent years ^ 

It was found that faculty recognized individual differences 
within each class by using the full nine, point scale, and that 
there were significant mean differences in ratings for each 
class, with Freshmen lowest and Seniors highest. Although these 
findings supported the intention of creating developmental 
descriptions of the characteristics, ratings across the charac- 
teristics were found to be very similar. 

In all phases of the development, and testing, of the instrument, 
very high correlations were fojjtnd ambng the characteristic . 
ratings.' Factor analyses of data -from the longitudi?*el sample 
on three separate occasions confirmed that a single underlying 
characteristic was Weing rated, rather than 'six performance 
characteristics as (differentiated by^ the handbook definitions. 
Trie longitudinal data provided evidence of significant improve- 
ment over time in ratings of two .entrance classes, 
corroborating the earlier cross-sectional evidence for a 
developmental measure! y 

The ratings were not correlated with other indices of student 
performance in Ithe college^ when background and program 
variables influencing these indices were controlled. This 
indicated tofeit faculty were not merely rating students according 
to their achievements in successfully completing the curric- 
ulum, There was also no relationship found between the Six 
Performance Characteristics rating and the Integrated Competence 
Seminar assessment developed by the college. More detailed , 
analysis of performance on the Integrated Competence Seminas 
(Assessment Committee/Office of Research and Evaluation, 1982) 
ra ised questions c oncerning t,he val idity of some of the Exe rcises 
in that assessment, however. 
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The Six Performance Characteristics rating facto^,was corre- 
lated with several measures of human potential administered in 
the college f s longitudinal study of stud4nt development 
(Mentkowski & Strait, 1983). These relationships supported the 
idea that the underlying dimension being ratecf by faculty was a 
general cogni cive-developmental continuum. Among the human 
potential measures*, the strongest pattern of associations with ^ 
the Six Performance Characteristics rating factor was found with 
a measure of Perry's scheme of intellectual and ethical develop- 
ment in the college years (Perry, 1970, 1981) 

Throughout the development and several cycles of analysis of the 
assessment technique, the faculty have discussed the' various 
possible practical applications of the assessment technique 
beyond its employment in the comprehensive program validation 
model. In Alverno's curriculum, narrative transcripts play an 
important role as a record of student progress. Faculty have 
found the rating process, and the data and written comments it 
has generated, especially useful in the preparation of narrative 
transcript material for stud£ntT£>! records. 

The faculty is continuing tfo experiment with /he assessment 
technique as this report goes to press. Further, and future, 
information can'be obtained from the Office oi Research and 
Evaluation or the Alverno College Assessment ^Sfcianittee . 
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INTRODUCTORY STATEMENT REGARDING CHARACTERISTICS 
OF ADVANCED STUDENTS 



During her first semesters at Alverno College 
the student develtys her competence in each of 
eight specified areas by focusing directly on the 
meaning, structure, and applicability of each 
competence within varied disciplines. As she 
gains assurance and experience in each competence 
area, ahe exercises her abilities more extensively 
in a variety of situations and, consequently, 
experiences' the interdependence of abilities with 
respect to each other and with respect to system- 
atic areas of knowledge, 

This experience of interrelatedn?ML with its t 
corresponding sense of personal internalization of 
abilities, becomes especially eviderft to hfer as 
she enters Into specialized study in her Sajor 
area, By exercising the range of her talents in 
developing^* complex abilities required by her 
major area, t:h^ student develops qualities which 
characterize her way of approaching tasks and of 
handling responsibility, ■ 

At the present time we are able to specify 
five such characteristics which apply to her t 
behavior - integrated, independent, creativ/, 
self -aware f committed - and a sixth - 
habituality - which modifies the others. The 
development of these characteristics takes place 



initially as the strives to acquire or to Improve 
abilities demanded by her discipline or profession, 
Gradually, the characteristics themselves become 
central to her style of working and to her exercise 
of personal responsibility, 

Because we think of these as contributing to 
personal and professional life apdH&cause we 
,believa in the possibility and importance of .making 
Judgments jabout "them, we incorporate them as 
general criteria in assessing student endeavor at 
advanced levels. As faculty assess students on 
designated abilities required^for the major, these 
six characteristics function as criteria. 

Also, Jjecause we believe these characteristics 
continue to develop in a person's professional and 
personal style of exercising responsibility, we 
incorporate assessment of them into our longitudinal 
evaluation program, Prior to graduation, faculty 
am :r evaluators make independent judgments 
on k senior's manifestation of these character- 
istic. Later, our graduates themselves and other 
appropriate individuals will utilize these criteria 
in juaging the degree to which the work of the . 
graduate as a professional and ..the graduate herself 
as a person manifest these characteristics'. 

.36," 
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DEVELOPING STl^VT 



A characteristic which is revealed by a growing harmonious relationship among one's 
in interacting with one's total bvlronnent. 

BEGINNING STUDENT ' • J * 

• Understands Antent on a quite literal 
level, Nuances, overtone*, and .connotations 1 
await further development 

• Senses relationships between various 
academic disciplines and abilities but 
largely relies 6n explicit examples of * 
this from external souKes,of authority 

• Needs explicit help in academic and personal 
goaNirectton and tlae-oanagenent 



INTEGRATI& 
(Wholeness, Unity, Soundness, Completeness) 

inner resources and which is manifested by congruent behavior 



* Describes relationships between various 
discipline^ seeks to coordinate disciplines 
that are^complemenrary and mutually supporting 



• Identifies with a single perception from an 
external authority and submits tn a somewhat 
passive manner to the directions of others in 
her learning process; needs frequent reinforce- 
ment from external authority in the accomplish- 
ment of her academic work 



• Perceives artfas of her life as compartmen- 
talized into ^multitude of worlds: family, 
school, leisure, politics, .nations, religions, 
ideologies; frequently iose^'Hho 1 am" as 
she goes from one .area o£ her life, to another 



• Describes se 



at the mercy of external 
I factors and may use one area of ' ■ Ufo 
that is viewed as rem 



escape an are 
imposed 

• Evldenc 
perforata: 
for total 
in collabo: 




# Recognizes the complexity of human issues pre- 
sented in the varies disciplines* (How do I 
reconcile the individual and coauwnity, the 
particular and the universal, law and liberty? 
How do I bring together feeling *nd thinking, 

Reason and imagination, subjectivity and objectiv- 
ity?) is willing to wrestle with this dialectic i 

• Evidences an awareness of the tension of 

, polarities in her own personal life as veil but 
is willing to live and work with the tension in-, 
a creative manner ^ 

♦ Describes self a* lively, adapt ir- 1 ro w ■ 

: a* "h factor 1 -ous 

,; : p* ' -J prof l >u 



r^onal concern for her own 
r than felt responsibility 
movement when she is working 
on with others 



OAOutlK STUDENT 



\ 



» Can take multiple perspectives on issues 
arising from the various disciplines and , 
from her own personal life experiences 



» Evidences a complementary dynamic between 
school, work, personal relationships with 
self as the Integrating factor * 

* 



wwlfutun 



• U concerned with her perarifSl future and 
sties herself linked inseparably with the 
future of the human community 



• Understands lln a Jilted way the connection 
between theory and praxis in her life 



Initiates gathering of personal Insights from • Begins to come across as a simple* lucid 



past' and present role, modeii of the total humsn 
community who have demonstrated by their \ 
behavior that they, have achieved integration 
between theory and praxis 
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interpreter of Life consonant with her own 
\ experiential background, Reliance on 
authority and tradition is meshed with a 
personal synthesis of her ovn lived experience 

Moves toward reiUiing ultimate meaning 
(transcendence) in the midst of the limiting 
boundaries of her life cycle 

Incorporates imighti f rots her personal put 
experience! Into hectare role within the v ; 



INDEPENDENCE 



The student vho operates -1th Independence, to a lesser or greater degree initiates,, develops, and carries through to ecapletlon 
experlences % relatlr S? to her toraal educative process, either academically, professionally, or personally, In soae way, she sets and 
achieves her owi educational, career, and life goals'in a sinner that reaches beyond the expressed requirements of her arta{s) of 
study; that is, she uses her ability' to Initiate and execute work on unfaallUr data and to apply this in new areas. She does this 
with a growing curiosity, enthusiast}, and energy. 



BEGINNING STUDENT 

e Manifests aspects of independent 
activity In discrete learning situations 
(e.g., chooses a topic froa a variety of 
offered, alternatives) . ■ 



DEVELOPING STUDENT 1 

# Initiates activity by responding to an 
open-ended stimulus (e.g.. selects air 
appropriate fraaevork for analyzing an 
arc piece) 



1 \Lf 



STUDENT 



« Manifests facility for initiating, 

pUnning, executing, and evaluating processes 

and projects related to her area and hc<; 
profession 



• Follows concrete suggestions and 
directions for specific instances in her 
educative process 



« Usually acts when something is called 
for or when she Is given something to 
take hold of (e.g., responds to sci:iull 
presented by the instructor) 



% 



« Advances plans with fever directions 
and/or suggestions (e.g., often suggests 
or designs subsequent steps vuhin a 
process) 

Perseveres toward an ia&aediate gcai in 
the face of obstacles 

# Bestlns tc extend the oodes of procedure 
learned In her academic life, to her 
professional and. personal life 



i Crows incuriosity and enthusiasm for 
learning that expresses itself In some 
interest and Involvement beyond the 
requirements and expectations 1 of her 
discipline (e.g/, relates concepts, or 
issueB from one area to another area) 



« On her ovn, tickles data on unfamiliar 
areas ad extracts from tsany and varied 
experiences to relate thea meaningfully to 
her inudlate educational ad professional 

experiences 



i On her ovn, takes hold of a situation; 
analyzes, organizes, and synthesizes it In 
relation to a self -chosen operational goal; 
and perseveres toward that goal education- 
ally, professionally, and personally 

a Reaches out, vl > spontaneous enthusiasm, 
beyond the formal »equlre«ents of her 
discipline and her profeialon for southing 
involving totally ; in her life 

Energetically and enthusiastically brings 
intoher educational , professional, and 
living experiences tilings she sees and 
hears chat others do not perceive • 



Developed bv 
Aftes«n( (Vwnlttee 
Alverno Ccl-U^e 
Milwaukee, Wisconsin 
Haw 1978 



i Demonstrate! an awueness of the inter- 
dependence o£ pooto by working vith others 
in giving and receiving (e;g., appropriately, 
shares or delegates authority and/or 
responsibility in group tasks) 
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CREATIVITY 



Creativity is the characteristic of the perwn which manifeats itself by meant of various creative styles, One such style is to produce 
original or unique Haas, Another equally valid style is to rearrange or uniquely synthesize the ideas of others without necessarily 
originating the*. Persons say be strong In one style or another or poeiesi aspects it both; These styles spring from i number of unique, 
individual characteristics such as imagination, flexibility, openness, inventiveness, etc. In order for these styles to be assessed, they 
oust somehow be demonstrated in the student's behavior, Therefore the. elements of sn individual's creative style including its develop* 
mental processes are stated here in tenu of behaviors which would result froo that, sty le(s). (The products of creativity ire fluently 
different across discipline sreai; however, the intent here Is to focus on some generic aspects of the creative proceas.) 



BEGINNING STUDENT 



• Produces responses which are original or 
4 unique, but not clearly focused around a 
central idea or goal, etc, 



e faquirts multiple examples of an idea f 
concept t method 



e Primarily has an affective (positive or 
negative) response to unanticipated 
circumstance* or ideas 

Deals with unanticipated circumstances, but 
may allow self to become sidetracked 

e Interprets and follows direct tons 
literally 

Relies upon step-bystcp means to reach 
a goal 



DEVELOPING STUDENT . 

e Begins. to perceive the value of producing 
a variety of ideas vleh respect to creating 
a product, reaching a goal, solving e 
probles ij 

Begins to perceive that the creative process 
should not be in end lb itself but should be 
y a means toward some end 

(Note: In both of the sbove, the atudent it ' 
learning about end how to use the learning 
structures as toots of the discipline as 
aids to the creative process.) 

e Generates similar .examples when given some 
examples of idea, method, etc. 

Extracts the concept when given an example 

e Begins to see that unanticipated circum- 
stances may not be obstacle; but opportuni- 
ties for the creation of new ideas 



/ e begins to perceive need for creating new 
options or ideas 

Perceives directions as guidelines but is 
able to go beyond them by creating nev 
options or ideas 



GRADUATING STUDOri 

o Internalizes the value and function of 
the creative process such that it is 
purposive, and evoked without outside 
direction 



e Can generate own unique examples; requires' 
a minimal number of examples in order to 
understand method, con:ept, etc. 



a Demonstrates flexibility when confronted 
vlth unanticipated circumstances or ideas 



a Kith mlnlcum direction can select and/or 
synthesize others 1 Ideas in s unique manner 
'or can generate her own unique Ideas 
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e Perceives creativity as a quality one 
possesses, e.g.! talent, rather than an 
ability one develops 
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e Begins to perceive that different 
creative styles exist and begins to identify, 
own preferred creative style 



i Recognizes own creative atyle *as well as 
the creative style of others 

Knows when and how to apply own creative 
style (including collaborating with others 
having different creative etylea whin 
appropriate) 

e h given situations cm articulate own 
creative proceas as well as defend it by 
producing s rationale 
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AWARENESS (OF CAP^IUTY) ^ 

A characteristic of the person which also includes as its "object" all of the person's other characteristics (independence 
creativity, etc.). !/ H 



4 



A characteristic which operates si a series of developmental stages within Jfche development of other characteristics. 



BEGINNING STUDENT 

^ % Has theoretical knowledge of some 
characteristics of competent and 
incompetent performance 

flaky. tentative and tenuous applications 
of abstract criteria to any aspects of 
her own behavidr that she can take 
seriously enough to momentarily look at 
< apart from herself \ 

i Experiences any evaluation of Herself 
as general affirmation or general 
rejection 

,i Hakes judgments or her own behavior 
. usually only after someone else points 
out concrete evidence for aer 



Y 



jlrw^itt her 



* "Does" activities as required 
academic program but does not identify 
with then 35 expressions of her own 
abilities that she can understand and 
develop 

• Tries to keep evaluation outside of 
herself 

/ 
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DEVELOPING STUDENT 

• Sees a criterion as something she 
either can do or can't do right now 

Begins to sense when her own performance 
in a given situation is uajmtially 
competent or incompetent ^ 



GRADUATING Site 



• Has so internalized the Idea of self- 
assessment (hat she practices it habitually 



a Identifies individual strengths and 
weaknesses that consistently appear In 
her performance 

a Begins to make judgments related to 
-behavioral observation 



o Begins to experience her own abilities 
in relation to her effectiveness or 
ineffectiveness in a given situation 



i Makes occasional, sometimes unrelated, 
decisions to develop some strengths and 
to eliminate some weaknesses and Ignore 
others 

Consciously makes occasional discrete 
relationships between her strengths and 
her personal and professional goals 



a Identifies Specific strengths and weak- 
nesses but puts them into perspective 



a In a given situation makes independent 
judgments about her general effectiveness 
which are congruent with those of others • 
who are careful, experienced observers* 

Hakes some reasonable predictions, in new 
situations, about whether 'or not she can 
develop abilities she* has not considered 
before but which she sees that this situa- 
tion calls for (can ask such questions as: 
What abiiitfes do 1 bring A this situa- 
tion? Whst do I know about this situation 
that will assist me to insure a competent 
performance? How is this situation 
different? Whit do 1 still np*d to learn 
about this situation?) ' \ 

i Articulates her own abilities in relation 
to each other, to their operation In differ 
ent contexts, and to experienced change 
in herself ^ 



a Makes thoughtful, consistent decisions 
about which aspects of her abilities she 
should consciously focus on for develop- 
ment and which she should not 

Experiences 1 the development of her 
strengths as the gradual achievement of. 
her personal and professional goals 
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• COMMITMENT # 

♦ * 

(SeiMnitia'ced Obligation, Involvement, Entrustraent, Reasoned Belief) 

v 

,A characteristic demonstrated by actively engaging oneself In finding and living out of a mission. The student undertakes 
'and completes activities or involvement in issues floated to her own beliefs t attitudes, and values; that is, those issues 
that encompass and surpass the "here and now.' 1 I 



BEGINNING STtaT 



DEVELOPING STUDENT 



GRADUATING STUDENT 



t focuses on issues directly affecting 
her personal life; sees Issue in 
relation to personal goals 



• Participates on her ovn Initiative 
(minitwlly— number of times a; well as 
responsibilities) in activities/ 
concerns fostering identification and> 
involvement in issues 



# Focuses on selective concerns/issues 
that are consistent with her beliefs 
and attitudes;* sees herself in relation 
to the issue 



• Selects concerns and Issues within . 
contexts of formal learning experiences, 
whether these affect her career choice, 
profession, or world environment 



• Involves herself In activities 
beyond her specified claps related 
work 



e Involves herself in concerns /issues 
that are larger Wan her specific 
career choice 



• Articulates concerns related to 
day-to-day situations (usually crisis- 
oriented) 



a Verbalizes concerns and involvement 
in personal issuerrelated to career 
choice 



e Tries to convince others of the 
importance of their involvement and 
articulation of concern 



• Respond! in a sporadic and not yet 
consistent manner to issues 



• Responds actively and positively 
to concerns 



e Prioritizes areas of concern and 
involvement 



• Responds to situations In which she 
is personally asked co participate 

# Responds to situations on a short- 
ten? basis 



a Responds with constructive suggestions 
for others to consider 

0 

a Expends significant time and/or energy 
in active participation in area of 
professional field and/or role as 
citizen 

Promotes activities/issues which 
support her View of the action needed In 
defined areas of her profession and life 
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Habltuailty is a dimension of 
of the person in several senses 

a) frequency, conslstency- 



HABITUALITY 



the other five basic characteristics of the person. Habltuailty modifies the other characteristics 



b) spontaneity, comprehensiveness- 

c) ease, perfection of operation- 

d) endu ranee — 



How often do the characteristics manifest themselves? To what extent are the 
characteristics predicta ble in a person's behavior? 1 

Tovhat extent is the source of the characteristics internal or external to the person 
(I.e., are the characteristics external expectations or internal needs)? 

When the characteristics are manifested, does the person experience positive or negative 
affective awareness? ' < 

■ # 

Do constraints serve Ss occasions for rallyiniifte other characteristics or as occasions 

where these characteristic?; wane? 



BEGINNING STUDENT 

> Is inconsistent in her exhibition of 
^the characteristics 1 of the per^m; 
frequency of denwnstrationj/deter- 
mined by factors outside fierself 



DEVELOPING STUDENT 



GRADUATING STUDENT 

• Exhibits the characteristics of the 
person frequently and consistently 



f Knows that; she can demonstrate or 
use^certain abilities bu>t has not 
associated these abilities with 
herself as a person 

Applies learned skills primarily in 
-the formal classroom seating 



t Responds to external stimuli 
(instructors 1 directions) rather 
than acting spontaneously 
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# 3$g^s to experiment with skills in 
new situations ■ 

Begins to generate more enduring us,es 
of particular abilities and transfers 
these. uses outside of format educa- 
tional settings 




• At times spontaneousiynjcorporates 
a^nuaber of skills In approaching a 

pi^fcletn 
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• Exhibits the characteristics of the 
' person in a wide variety of contexts 
(academic, pre*professional, social,, 
and personal) 



Has developed a set of internal 
standards for her behavior 

Uses these standards to help her strive 
for quality and professionalism and will 
• gain an affective dissatisfaction with 
failure to meet these internalized 
standards whenever possible 

Exhibits the characteristics of the 
person, even in Ar-fcke of contradictory 
external pressu/es . , 
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Pilot Study ofNSix Performance 
Character is tics Rating , 
Spring 1978 
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Presentation by the Office of Research and Evaluation to the 
0 * Assessment Committee on th6 pilot ■ 

study of the^Six Performance Characteristics Rating 



The relative frequency distribution of students who wejre rated 
on the Six Performance Characteristics are presented in graphic form. 
The graphs .indicate that the majority of students c luster (apeund / 
the Developing arid Graduating points of the scale (5-9) , /bu^^tjie 
patterns of the distribution varied across characteristic 
Habituality and Integration followed almost "an identical 
Independence and Commitment were distributed in a somewhat similar 
pattern .and so were Creativity and Self-Awareness (sen Figures 1 
and 2). 




/ 



A table of inter-correlation of all Six Performance Characteris- 
tics As presented (see Table ,1) . Correlations range from .59 to 
.78. All were significant at the .001 level, indicating that all 
Six Performance Characteristics are^highly interrelated. 

Consistency of f aculty^udgments (inter-student consistency) 
was examined by inves£igabing faculty ratings across divisions. 
Do faculty numbers in a particular division tend to^rate students 
higher: or loiter? Students' mean ratings were examined in the 
following divisions: Arts and Humanities, Behavioral Sciences, 
Education, Mathematical and Natural Sciences,* Nursing and 
Performing Arts (see Table 2). ^ 

The final multivariate statistical analysis indicated that 
Education students were rated significantly higher than Nursing - 
students within three characteristics: "Independence, Creativity 
and Self -Awareness. 

Based on this analysis^one cannot conclude that faculty members 
tend to rate higher *m a particular division. The significant 
difference obtained may be due to the fact that 33% o& the 
Education students as apposed to only 24% of Nursing/Students 
were awarded Honor 5^. Education Honor students may have raised 
the means of the ratings. This may support the assumption that 
higher ratings in the divisions may be due to higher performance 
of students rather than differences in faculty rating. 
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Figure I. Frequency distriburions of Six Performance Characteristics 
rating for 1978 graduating seniors. 
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FREQUENCY DISTRIBUTION OF STUDENTS 
RATED ON THE SIX CHARACTERISTICS 



INTEGRATION 
HAS, ITUALITt 



INDEPENDENCE _ 
CORRTTMEST .1 . 1 
CREATIVITY • » , 
SELF AWARENESS 




BEGINNING 



9 POINT SCALE 



CRADUATIKG 



Figure 2. Combined frequency distributions of Six Performance 
Characteristics ratings for 1978 graduating seniors. 
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Inter-correlation Coefficients of the Six Performance Characteristics . 

Ratings for 1978 Graduating Seniors I 
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Note. All coefficients in the table are statistically significant at the ,001 level. • 



Table 2, 

Means and Standard Deviation of. Six Performance * 
Characteristics Rating for .1978 Graduating Seniors Across Divisions 
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7.500 


7.000 


Sciences 




SD 


.753 


1.506" 


.894 ' 


1 366 


1 178 

X i J 1 \J 




Education < 


9 


is 


7.889 


8.333* 


•8.111* 


8.222* 


7.667 


7,889 


* 




SD 


.782 


' .707 


1.054 • 


. .833 


1.323 


1.054 


Mathematical and 


2 


M 


7^00 


8.000 


6.500 


6.000 


7.000 


7.500. 


Natural Sciences 




SD 


. 707 . 


1Mb 


.707 


.000 


2.828 


2.121 


Nursing 


33 


}f 


7.121| 


7.121* 


6.909* 


6.727* 


7.152* 


7,061 






SB 


1.053 


1.166 


1.128 


1.376 

<> 


1,-253 


1.688 


Performing Arts * 


13 


M 


7.769 


7.846 


7.692 


'7.692 


8.385* 


7.462 






SO 


1.301 


1.144 


.947 


.947 


1.044 


1.127' 



5.6 



7.11 



7.0 



6,7 



7-. 2 



Significant differences at, the .01 level 



So far the data have indicated consistency of inter-student 
^ratings (judgment ^among students)' and cSnsistency of intra- 
istudent ratings (judgment of one student across Six Perform- 0 
ance Characteristics), 

• The Office of Research and Evaluation then 'investigated the 
relationship between the Six Performance Characteriatics and ' 
.indicators of performance in the Alvejrno learning process » 

^J^b well as their relationship to a cognitive-developmental 

measure (Loevinger's Ego Development Measure). Such an analysis 
may assist faculty in understanding what aspect of the students' 
development is being measured by the Six Performance Charac- 

• t eristics: ' 

• Do the Six Performance Characteristics' measure 
developmental changes in the person? 

• Are the characteristics reflective of performance . 
within the ei^ht competences? • ^ 

• Or both? 

* • 
The measure of rate of validation was chosen as a way to * 
examine student performance within the learning process 
through levels 1-4. The total number of validations (77) 
expected to be completed at t f he ^>eginnin£ of Junior year 
was taken as the maximum measure of rate of validation. 
Only students who had completed 6 semesters at Alverno were 
included. \ 

When rate of validation was correlated with each of the Six 
Performance Characteristics only Habituality was signifi- 
cantly correlated (p<\04). One should keep in mind that 
the* measure of rate of validation reflects student perform-* 
ance through levels 1-4 whereas the Six Performance Charge- *, 
teristfes reflect student performance also at the mora 
advanced levels. This analysis did not support the idea , 
that che rate* of validation from levels 1-4 predicts per-% ' 
formance at levels 5 and 6 "as, measured by these character- 
istics (see Table i) . 
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T,sMe \ « 



Correlation Covi i ic Jonts BeS.ween Kale oi Validat ion 
rind Si;< Peri e nuance Chirac tor i^t irw Rat i n» 
fur 1978 (IraduaihiK Seniors 



[nct-^ra- J side pen- '.'ro-wiv- Self- Commie- \\;ihi i u 

oration dent:e ;• : \ Awareness went ..alitv 



Race of 
Val idat ion 



. v, n 

. 1.62 



•l If- 



" to? 



. 1298 
. |V2 



. J31 



Note. Total number ot va tidal i on?- in all fuwr levels •• 'H 

A 1 1 students wore i n rhe tr' Junior year Spr i ! ^ 1 
All A? students had six semesters at Alverno 



The Six Put S vtrnaiu'.e .Character i st ies were not si?*nif ieatu ly 
related to Loevlu&er * $ \U\o Development Measure' (;see Table 4) • 
Similar results were obtained vhen the Six Performance Charac- 
teristics were correlated with a*:;e (see Table 5)- Further 
measures at" the learning process and other developmental 
measures may he used to further refine the meaning of the Six 
J *e r 1 u rm a n c e (lha r a t: t e r i * ?. s e s , 



Tab le « * 

Correlat Son Between Loev inker's- Kgo Development 
Measure and the Six Per formanre.. Charac j.erist i.es Hating j 
of Spring 19 78 Graduates (a~ 



Integra- tndepon- Creat iv- Sel f- Cosmni i - Habitu* A virago 

tlop dence_ iiv ; Awareness roent.. ty Score i 

Loevinger's r* . .01 72 , 1130 *05t2 .1925 .1222 .2739 .1453 
Ego Develop- 
ment Measure p=- .463 .234 . 390 . 3.46 .233 ' *065 .213 
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x ViJY formaiu'o Character l st i c 
{ l )?S Cradiiaces (n-ln j 



I rule pen- 
deJU'i: 



l.'lHMt iv~ 



Soli - 

Aw a r en ess 



merit 



Commi t - 



an t. v 




0 KM 



4f)H 



Another way o? validating the neasu recent of the Six 
i'er: orinaiu-e i'asaras'ter i ss: i i ■; ? .-. fxaidnr r.jiitn'.s of honor 
studezus, Do Honor students de-^nstrato si^n'i t" icantly 
higher ratings across ail Six i'er f ori'sam-e Characteristics/ 
Mean ratings of Honor students were compared (see Table b) 
.. Ith mean ratings of students who did not achieve honors* 
Kosul t s obCnHLtjed d id i. nd i rale i hat honor s t udents wen 
rated s I y.ni f u* an 1 1 y higher on each of the eharnc t ev i st i c s , 
hot t lu ?a*:t t hat they were rated after being nominated 
cannot he ovei' looked ■. 




A 6 



Ta 



Honor f£)C ihj^s Compared to No Honor Rat In;.; 
on M a oh of S i. x 1 ' e r f u r a nee Char a c L o r 1. stir. 



Charac ur : st i< 



Honor / 
No Houoi 



Number 



Moan 



Standard' 
Dcv-iat ion 



Average Kat i.n>> 
Across Six 
Character is t i < 



honor 
No Honor 



8.6 307 
6.9149 



44420 
94281 



6. 58- 



mu.v.rat J on 



«iu nun* 



S ,571 4 

6 . .Hi V ; 



. 64fc2 1 
1 .06? 3 



Independent! 



Honor 
No Honor 



14 

31 



8. 7143 
7,0980 



4 , ** :> 5 



Croat Ivi tv 



Honor 

No Honor 



14 
51 



8.4 286 

6.84*1 



.64621 
! . 06 5 'J 



7 q ;V 



So IT -nwarentM 



Honor 
No Honor 

Hon or 
No Honor 



14 8.6429 .6.5332 6.09* 

51 6.6078. .1.201 ) 

14 8.7857 .57893 4 6f)ft 

51 7.0980 1/3154 



Haoitua 1 it: v 



Roiv r 
No Honor 



8*6429 
6.862 7 



.49 725 
.91694 



6. 96 



* All t valuos are significant at the .001 level. 



\ 
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Several quest: ions were raised by the Assessment Committee 
while discussing the data: 



• Are the definitions of the characteristics adequately 
differentiating between each of the characteristics; 

# Should students he rated by faculty both from the ma.jos 
and minor areas to improve the quality of the ratings".' 

* At what point in time should the rating be done 
nominal ions for honors? • v^** 



V 
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• When rating is done as a departmental activity, are 
students rated by faculty members who used to teach 
the students in their Freshman or Sophomore year but 
who are not familiar with their performance in the 
upper levels? by rating each year the process will 
be simplified, It will be an ongoing process in which 
faculty members wilt judge current performance. 

» Can we obtain feedback- from faculty members on how 
they judged students? What .was I he basis lor judg- 
ment ? 



• 



It may be better to ehan^ the definitions rather 
than change tlie number of characteristics. 

To what extent did faculty members use the defini- 
lions '.'line rating'* How often are individual con- 
ceptua Ligations of the Six Characteristics used? 

• Facility should be asked to rati? each class this 
spiting. More data, are needed before we begin 
V reviewing the definitions of the charaexerist ics v 
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Name 



Please rale this st udent on each of the fol lowi n^ s i :•. char 
Place a q/heck («/) in the space desired. 



Hie scale is divided into 3 stages: 

I 2 3 4 5 6 7.89 

4 — 1 — rH — i — I f~~T~4 



U Bt^inni :r; student; i) = Developing StiVkni; C - Graduating. 



]^itcgr,at ion 
Independence 
Creativity 
Awareness 



ft 7 8 9 



1 2 
I 



•', 6 5 6 7 8 9 

1 4tH H — I — I — I 



7 8 9 



i I ! I hi 



1 2 3 '4 5 6 7 8 9 

i \ I i ! Ill- 



3 A 



6 7 8 9 







1 — 1 


hH — 1 


Commitment J_ j_ j 


h- 4- H 







1 2 3 



5 6 7 8 9 



|- I I: H — I — I- — I — i — \f\ 



Habi tuality 



Note: Kabituality modifies the other characteristics v 

i.e., habitually committed, habitually independent, etc 



5/78 
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t APPENDIX C 



Revised Six Performance 
Characteristics Rating Form, 
Spring 1979 
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SIX PERFORMANCE CHARACTERISTICS RATING 



Assessment Committee/Office of Research & Evaluation 
ALVERNO COLLEGE 



J 



Funded by a grant from the National Institute of Education: 
Careering After College: Establishing the Validity of Abilities 
Learned in College for Later Success 
(N I E-G -77-0058) 
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Austin Ooherty 

Alvemo College 

3401 South 39th Street 

Milwaukee, Wisconsin 53215 
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© Copyright 1978; Revised 1979. Alverno College Productions, Milwaukee, Wisconsin. All tiqhts reserved under ! 
International and Universal Copyright Conventions. Reproduction in part or whole by any method is prohibited by I 



Name 



Instructor 



Cl^z'le one: Fr So Jr Sr Course 

Major: 

Plecise - ate this student on cm h of the following six characteristics, ^lace a check J/) 
in the p.ice* des Lrtci. i 



1 2 3 4 l/ 5 6 7 8 9 

i • 1 1 M 1 1 1 1 . 

Beginning ' ^ Graduating 

\ ( 

• While the definitions of the s *_x, characteristics describe the Beginning Student, the > 
Developing Student and the Graduating Student, please d o not limit your rating to 

u ~" for Freshmen, !, G" for Seniors, etc. Use the entire 9-point scale for each student. 

• Study the definitions of all the characteristics (see Jb^Tculty Handbook on Learning and 
Assessment). We recognize that you will be rating the whole person. However, try to 
distinguish between each of the characteristics when rating an individual. 

• The last page allows space for comments. Comments will provide an ongoing record of 
developmental performance that can assist faculty in preparing the narrative transcript 
when the student graduates. 



Thank you for your contribution. 



The Assessment Committee 



1 



Integration 



Habituality 



12345 / 6 789 

MIN IM -Hi 



4 

1 2 3 45 6 7 8 9 

1 i r 1 1 1 1 1 1 1 



Mote: Habituality modifies the other characteristics, i.e., Habitually Committed; 
Habitually Independent; Habitually Creative; etc. 

If you have rated a student at 4 or above on this characteristic, then indicate 
on the Habituality scale the extent to which you have observed tha^: this 



characteristic is habitual . 
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123456789 



Independence ^ j j J j |_ 



Habituality 



1 2 3 4 5 6 7 8 ^9 







f— 1 — ! — 1 — i 




4— 4— 1 — 1 


1 — I 







Note: Habituality modifies the other characteristics, i.e., Habitually Committed; 
Habitually Independent; Habitually Creative; etc. 

If you have rated a student at 4 or above on this characteristic, then indicate 
on the Habituality scale the extent to which you have observed that this 
characteristic is habitual. 



Creativity 



Habituality 



1 2 3 4/ 5 6 7 \8 9 

-+ I" i \\ 1 1 1 h~ 

: . . •/ 
( 

123456 789 

1 I I I — h- 




Note: Habituality modified the other characteristics, i.e., Habitually Committed; 
Habitually Independent; Habitually Creative; etc. 

If^yoW-ftave rated a student at 4 or above on this characteristic, then indicate 
Tfe Habituality scale the extent to which you have observed that this „ 
icteristic is habitual. 
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Awareness 



1 — I — | — I — I — i — I — I — b 



1 2 3 4 5 6 7 8 9 



Habituality 



f-H — hH — hH — I — f- 



Note: Habituality modifies the other characteristics, I.e., Habitually Committed; 
Habitually Independent; Habitually Creative; etc,. 

If you have rated a student at 4 or abovyd this characteristic, then indicate 
on the Habituality scale the extent to^ch you have observed that this 
characteristic is haMtual , 



7 



CoOToitroent 



\ — i — hh — y 



Habituality. 



-) — HH — I — I — I — I — \ 



I 



Habituality modifies the other characteristics, i.e., Habitually Committed; 
Habitually Independent; Habitually Creative; etc. 

If you have rated a student at 4 or above on this characteristic then indicate 
on the Habituality scale the extent to which you have observed that this 
characteristic is habitual. NOTE: A PAGE FOR COMMENTS FOLLOWS THIS 

PAGE IN THE INSTRUMENT. 
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APPENDIX D 



Six Performance Characteristics 
Rating Procedures, Spring 1979 
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Six Performance Characteristics 
Rating Pt^oce jures » Spring L 9 7 9 

t . , / 

Mix. students in tne Weekday Collect will need coHJe rated for 

us to further understand the developmental nature of "the 

characteristics. 

t 

In order to obtain more reliable? ratings, both the student's 
major and minor department should be asked' to rate her in 
her Junior and Senior yejfrf on the Six Performance Character- 
istics,. 

' Last year departmental consensus* ratings for ..majors were 
obtained from at: least two faculty ratings- , We did not 
collect individual faculty ratings that were made before the 
consensus rating. This year, in order to add to our under- 
standing of the role that expert judgment plays in the 
ratings,- we need individual ratings first from at least two 
faculty. These faculty should be chosen on the basis of how 
closely they have worked with the student. Then the depart- 
ment (or faculty group) provides a consensus rating, during 
which time the faculty who filled out individual ratings 
would jointly fill out a separate consensus form. Both indi-J 
vidual ratings before consensu*^ and the final consensus 
rating would be collected. * 

Freshmen and Sophomores need to be rated by at least two of 
their instructors. The ratings would be averaged for this 
group. Consensus ratings would not be attempted for this 
group because of anticipated difficulties in creating time 
schedules for instructors to get together. 

In order to coordinate the ratings of the Freshmen and 
Sophomores, certain faculty with Freshman and Sophomore 
classes will be asked to rate their Freshman and Sophomore 
students. Since the faculty on the Assessment Committee 
are most familiar with the meaning of the Six Performance 
\ Characteristics, they willVfirst be asked to rate, Other 
faculty will then be chosen^ to rate those students not 
covered by Assessment Committee members* 

Given our need to prepare -narrative transcripts at the end of 
the Senior year, comments recorded during the earlier years 
will provide us with some information that would be extremely 
helpful in creating a developmental picture of student's 
progress and outcomes for the narrative transcript. Thus, a 
comment sheet will be added to the rating form for this 
purpose. 

In order to get underneath the. sources for the expert ju.ig- " 
ment faculty use in rating, w/j will ask faculty who Vated 
students to respond to a short- questionnaire concerning the 
sources for their judgment after all ratings are completed. 

1 61 



s 



67 



Fa-::h chai r pe rson will receive Liu. 1 : ' I low i n>; narerials from the 
Assessment Corrmi t; r ■.■<- r ^ . 

1. Materials, to rate all :pajor and I nor students i*n your 
department. ■ ( 

a. List of all. major "and minor students in your department: 

b . K n o u g h rat i n g !' o r m s f < > n e a e h Junior and S e n i o r s t ud e n c 

fur both individual and consensus rat ing^ Cor* major 
on 1 y * * M i no r s t uden t s r a t ed one e on fcy . ^ 

A I ; » ; j I n s c rue t o r s i vonr Division who wi i. 1. be ask^d to 
ra t e. s tudent.s who a re Fre:#ff!yn and Sophomores . 

a. A class list for each instructor who will be asked to 

rate. ' 

b . K n ou h r a t: i n £ f o r ■*: t s for e a e h i. n s t r u c t o r , 

3. Questionnaires tor the faculty on their use of expert * 
judgment. 

During Faculty institute this spring, the Assessment Committee 
will explain the rationale and procedure to the faculty. Thy 
Chairpersons will then be, asked to coordinate the scheduling of 
time within "the Division so" that the rating can he accomplished 
before May 25, 1979- f 

iftank you for your assistance in this worthwhile endeavor. \ve 
expect to be able to present the results in tfU fall Faculty 
I nst i tute , 



^EacK major student is rated indivicfunijy by, two faculty. 
Those two faculty then come to consensus on a third form. 

"Each minor student is rated bv the one fjacultv member in 
the department who "knows' 1 the student best. 
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x P e r f o nil a n c o Characteristics Rating 
Faculty Questionnaire , Spring 1979, 
With Summary Data 
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HOW DID FACULTY RATE THE CHARACTERISTICS? 

Purpose: We are interested in how the faculty rated. What was the basis for judgment? 

At this point in time, we expect that the ratings are an accurate indicator 
of faculty expert judgment, but we need clarification on what tine various 
sources^f this expert judgment are. 

Please respond to the following sources; \ 

i. Check the box next to each source you used, intuitively or analytically, to make \ 
judgments* Then rank order the sources you checked with "i" as most important, 
11 2", as second. most important, etc. There is space next to the source to comment 
on how it was used . 

Check 

If Rank 

Used Order 1 
I0t);:' 1 observed behavior 



9 7 Z :.! • performance on assessment techniques 
932 3 conversation with student 



100% 3 your ""sense" of the student 



2!?^ JL definitions of the characteristics 

y 

others plekse list 1* 



\ 



3. 



How clear and understandable were the definitions: 



Integrat ion 


Low 
L 


2 


. 3 


k 


5/ , 


6 


High 
7 


. M « 


4 . 7 


Independence 


! 


2 


3 


'-T 


i 


b 


7 


M - 


5 . 0 


Creativity 


■ I 


2 


3 


4 


* 5 


6 


** 

1 


M * 


4 , 7 


Awareness 


l 


2 


3 


4 


5 


b 


7 


M « 


4 . 7 


.Commitment 


i 


2 


3 




5 


<> 


7 


M * 

y 

»^ 




Habitual ity 


l 


2 


3 


4 : 


5 


u 


7 





What needs to be done to make this a more effective process? 



4. How was consensus reached for the group of students you rated? 



Alvcrno College r >y , 

Assessmewt Center 0579 f>r> 
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Simple Correlations Between Human Potential Measures 
'and the Six Performance Characteristics 
Rating Factor in the Longitudinal Study 
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APPENDIX F Table 1. 



Simple Correlations Between Entrance Assessments 
on the Measures of Human Potential and the 
Six Performance Characteristics Rating Factor 



Six Performance Characteristic Factor 



Human Potential 
Measure 
at Entrance 



Two and One-Half Years 
After Entrance 



Three and One-Half Years 
After Entrance 



Measures of Vocational, 
Educational and Personal 
Issues • 

"Best Class" Essay 

"Decision" Essay 

"Career" £ssay 

Sentence Completion Test 

De f tiling Issues Te s l 
V7, Score 
i) Score 

Test of Cognitive Development 

Picture Story Exercise 
Receptive 
Autonomous 
Assertive 
Integral ive 

Self-Def in» tion 

Ach ievement Mot ive 
Affiliation Motive 
Power Mat ive 

Learning Style Inventory 
C o n c r e tc K x p e r i e n c e 
He f I f*c t iv-e Obse r va t Ion 
Abstract 

Concep t ua 1 i z a t i o n 
Act ive Experimentation 
A b st ra c t / Co n c ret e 

Le a r n i n g 0 r i e n t a t h 
Act t ve/ RefJ ec t ive 

U*a i n I m\ T)r J en t a t i n 



. 08b 


(120) 


.157* ■ 


( 123) 


.053 


(121) 


.123 


(124) 


.056 


(109) 


.087 


(109) 


-.11 J 


(120) 


'-.075 


(123) 


. 10 5 


( 1 0/* ) 


. 134 • 


(107) 


.05 3 


(104) 


. 1 39 


( 1 0 7 ) 


-.022 


(118) 


-.0f9 


(121) 


-.036 


(120) 


-.010 


(123) 


.07 3 


(120) 


.114 


(123) 


-.007 


(120) 


-.004 


(123) 


-.225** 


( 1 20) / 


-<208* 


(123) 


-.081 


(120) 


-.173* 


(123) 


- . 05 3 . 


(120) 


-.051 


(123) 


-.060 


(120) 


-.037 


(123) 


. 0 1 7 


(12 0 


.028 


(12 3) 


. L05 


(12*) 


.1 24 


(126) 


.183* 


(124) 


-.2 50** 


(126) 


- . 0 3 7 


( 124) 


-.14 J 


(126) 


-.012 


'(124) 


.124 


(136) 


-.083 


( 1 24 ) 


~. 164 


(126) 


. 106 


< 1 24 ) 


.222** 


(12!-. 
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Table 1 . continued 



Human Potential 
Measures 
; at "E ntrance 

Test of Thematic 
Analysis 

Critical Thinking 
Appraisal 

inference 

Recognition 

Deduction 



S i x P e r f o rnv an c e Cha r ac t e r i s t i c Factor 1 _ 

Two and (hie-Half Years Three and One-Half Years 



After Entrance 



-.141 (122),. 



138 (115) 
187 (115) 
113 (115) 



After Entrance 



,079 (S25) 



• 101 
,105 
-.041 



f 118) 
(118) 
(118) 



■ *i' ■ .05 
**!■» .01 

'Nutfc: Numbers in parentheses art 1 sample fcizi 
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APPENDIX F Table 2, 

Simple Correlations Between Second Assessment 

on the Measures of Human Potential and the 
Six Performance Characteristics Rating factor 



Human Potential 
Measure 
at Entrance 

Measures of Vocational,' 
Educational and Personal 
Issues 



Six Perfo r mance Cha r acteristic Faccor 

Three and One-Half Years 
After Entrance 



Two and One-Half Years 
After Entrance 



"Best Class" Essay 


.232** 


(120) 


.195* 


(123) 


uecision essay 


. 109 


(121) 


. 14 7 


(124) 


"Career" Essay 


.186* 


(120) 


.082 


(123) 


Sentence Completion Test 


.28l***(120) 


. 181* 


(12 3) 


Defining Issues Test 










PZ Score 


.225** 


( 1.06) 


.258** 


(1 10) 


D Score 


. 163* 


(106) 


.288*** (110) 


Test of Cognitive Development 


. 218** 


(118) 


-1 .312** 


*U21) 


Picture Story Exercise 










Receptive 


-.3 32*** (120) 


. 103 


(123) 


Autonomous 


.053 


(120) 


.193* 


(123) 


Assertive 


. 188* 


(120) 


.204* 


(123) , 


Integrative * 


.031 


(120) 


-.031 


(123) 


Self-Definition 


-.068 


(120) 


-. 138 


(123) 


Achievement Motive 


- . 024 


(120) 


.012 


(123) 


Affiliation Motive 


.011 


(120) 


-.140 


(123) 


Power Motive 


. 090 


( I 20-H i 


. 09 1 


(123) 


Learning Style Inventory 










Concrete Experience 


- . 080 


( i 24 ) 


-.048 


(126) 


Reflective Observation 


-.12? 


(124) 


-.258** 


(126) 


Abstract 


.089 


(»24) 


.022 


(126) 


Conceptualization 










Activd*t£xperlmentation 


.036 


(12/) 


.132 


(126) 


Abstract /Concrete 


.096 


(124) 


. 038 


(126) 


Learning Orientation 










Active /Reflective 


.09 3 


(124) 


.223** 


.(126) 


Learning Orientation 
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Table 2. continued 



Six Performance Characteristic Factor 



Human Potential 

Measures 

at Entrance 

Test; of Thematic- 
Analysis 

Critical Thinking 
Appraisal 

Inference 

Recognition 

Deduction 



Two and One-Half Ye^rs 
After Entrance/*" 



.0*3 (122) 




140 (115) 
,212* (115) 
,304***(115) 



Three and One-Half Yearsi 
After Entrance \ 



-.044 (125) 



.111 (118) 
.17** (118) 
(118) 



**Ap 



,05 
.01 
.001 



Note; Numbers in parentheses ;irv sample hi^-s, 
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APPENDIX F Table 3. 

Simple Correlations Between Third Assessment 

on the Measures of Human Potential and the 
Six Performance Characteristics Rating Factor 





Six 


Performance 


Characteristic 


Factor 


". 

Human Potential Two 
Measure 
at Entrance 


and One-Half Years 
After Entrance 


Three and One-Half Years 
After Entrance 


Measures of Vocational, 


s — 








Educational and Personal 










Issues 










"Best Class 11 Essay 


.243** 


(121) 


.137 


(124) 


"Decision" Essay 


,370***(120) 


.363** 


*(123) 


"Career" Essay 


.253** 


(121) 


.218** 


(124) 


Sentence Completion Test 


. 212** 


(120) 


.092 


(123) 


Defining Issues Test 










P% Score 


.361***(102) 


.290***(105) 


D Score 


.326***(102) 


.282** 


(105) 


Test of Cognitive Development 


.215** 


(113) 


.195* 


(121) 


Picture Story Exercise 










Receptive 


.034 


(119) 


.005 


'(122) 


Autonomous 


.084 


(119) 


-.097 


(122) 


Assertive 


.021 


(119) 


-.089 


(122) 


Integrative 


-092 


(119) 


y 040 


(122) 


Self-Definition 


.017 


(120) 


-.022 . 


(123) 


Achievement Motive & 


-063 


(120) 


.096' 


(123) 


Affiliation Motive - 


. 1.1.8 


(120) 


-.268***(123) 


Power Motive 


.016 


(120) 


.055 


(123) 


Learning Style Inventory 










Concrete Experience 


.019 • 


(124) ■ 


-.053 


(126) 


Reflective Observation 


.079 


(124) 


-.199 


(126) 


Abstract 


.105 


(124) 


.161* 


(126) 


Conceptualization 










Active Experimentation 


.093 


(124) - 


-.031 


(126) 


Abstract/Concrete 


.076 


(124) 


.127 • 


(126) 


Learning Orientation 










Active/Reflective 


.002 


(124) 


.104 


(126) 



' Learning Orientation 



73 



ERIC 



76 



Table 3. continued 
- * 



Six Performance Characteristic Factor 



Human Potential 

Measures 

at Entrance 

Test of Thematic 
Analysis 

Critical Thinking 
Appraisal 

Inference 
Recognition 
* Deduction 



Two and One-Half Years 
After Entrance 



.103 



(122) 



Three and One-Half Years 
After Entrance 



,105 (125) 



,21-6** (115) 
.276***(115) 
.260** (115) 



,124 
,143 
.103 



(118) ■ 

(118) 

(118) 



*P < .05 

**p < .or 

***P < .00.1 



ERIC 



Note: Numbers iff parentheses are sample sizes. 

\ 74 
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